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An account is given of the methods of working of Experimental High Energy Particle Physics, from the viewpoint 
of statisticians and others unfamiliar with the field. Current statistical problems, techniques, and hot topics are 
introduced and discussed. 



1. Particle Physics 
1.1. The Subject 



Particle Physics emerged as a discipline in its own 
right half a century ago. It pioneered 'big science'; 
experiments are performed at accelerators of increas- 
ing energy and complexity by collaborations of many 
physicists from many institutes. It has evolved a re- 
search methodology within which statistics is of great 
importance, although it has done so without strong 
links to the statistics community - a fault that this 
conference exists to remedy. Thus although a statis- 
tician will be familiar with the research methods and 
statistical issues arising in, say, agricultural field trials 
or clinical testing, they may be interested in a brief 
description of how particle physicists do research, and 
the statistical issues that arise. 

Particle physics is also known as High Energy 
Physics 1 and the names are sometimes merged to give 
High Energy Particle Physics. Whatever it is called, 
its field of study is all the 'Elementary' Particles that 
have been discovered: 

• The 6 quarks (u, d, s, c, b, t) 

• The 6 leptons (e, /i, r, u e , v ^ , v T ) 

• The intermediate bosons: W, Z, 7, g 

• The 100+ hadrons made from two quarks( 7r, K, 
Z? s (2317)... ) or three quarks (p,n, A...) or five 
quarks (0 + ---) 

To this long list must be added the corresponding list 
of antiparticles. However this is not all: the domain 
of particle physics also includes all the particles that 
have not yet been discovered - some of which never 
will be discovered: 

• Higgs boson(s) 

• squarks and sleptons 



lr rhe terms are almost equivalent; strictly the phrase 'High 
Energy' means 'above the threshold for pion production', i.e. 
the energy at which a collision between two protons can produce 
three outgoing particles. 



• Winos and Zinos/charginos and neutralinos 

• further hadrons 

• etcetera etcetera... 

This list of proposed particles is limited only by 
the imagination of the theorists who propose them - 
which is no limitation at all. 

For each species of particle we want to establish: 

• Does it exist? 

• If it does exist, what are its properties: its mass, 
its lifetime, its charge, magnetic moment and so 
on? 

• If its lifetime is not infinite, what particles does 
it decay into? What are the branching fractions 
to different decay modes? What are the distri- 
butions in the parameters (energies and direc- 
tions) of the particles in the final state? Do they 
agree with our theoretical models? 

• What happens when it collides with another 
particle? What processes can occur and with 
what probabilities (expressed as cross sections)? 
What are the distributions of the parameters of 
the particles produced? Answers will depend on 
the target particle and the collision energy. 

1.2. Template for an Experiment 

To study some phenomenon X, which could be 
any of the above, a particle physics experiment goes 
through the following stages: 

• Arrange for instances of X 

This may involve a beam of particles, directly 
from an accelerator or through some secondary 
system, striking a target; the beam and tar- 
get particles and the energy are chosen as being 
favourable for X. It may entail a colliding beam 
machine like LEP for the Z or BABAR for CP vio- 
lation in the B system or the LHC for the Higgs. 
It may be done by producing particles and then 
letting them decay, as in the studies of CP vio- 
lation in the K° system. An extreme example is 
proton lifetime studies, where one just assembles 
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a large number of ordinary protons (perhaps as 
hydrogen in water) in suitably low-background 
conditions deep underground and waits to ob- 
serve any decays. 

For important studies dedicated experiments 
(even accelerators) are built. For many more, 
the experimenter utilises data taken with an 
experiment designed primarily for another pur- 
pose but also favourable for X. An example 
is the study of charm mesons at BABAR, Belle 
and CLEO, for which the primary purpose is B 
physics. 

• Record events that might be X 

A detector is built (or an existing detector is 
utilised). 'Events' - interactions or decays - are 
observed by a whole range of detectors (track- 
ing detectors like drift chambers and silicon de- 
tectors, calorimeters that measure deposited en- 
ergy). Fast logic and/or online computers select 
the events that look promising, and these are 
recorded: the phrase 'written to tape' is used 
even though today the recording medium is gen- 
erally disk storage. 

• Reconstruct the measurable quantities of the 
visible particles. 

The electronic signals are combined and inter- 
preted: points are joined to form tracks, and 
measurement of their curvature in a magnetic 
field gives the particle momentum. A calorime- 
ter may give the energy, a Cherenkov counter 
the velocity. From this emerges a reconstructed 
'event' as a list of the particles produced, their 
kinematic quantities (energies and directions) 
and possibly their identity (as pions or kaons 
or electrons, etc.) 

• Select events that could be X by applying cuts 

Knowing the pattern one is looking for, one 
can then select the events that contain the phe- 
nomenon being studied. 

A key point is that this selection (and also the 
electronic selection described above) is not going 
to be perfect. There will always be a selection 
efficiency which is less than 100%. 

There is also a chance that some of the events 
that look like X and survive the selection and 
the cuts are actually from some other process. 
There will be a background which is greater than 
zero. Statistical techniques are obviously im- 
portant for the treatment and understanding of 
efficiency and background. 

• Histogram distributions of interesting variables 
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Figure 1: Examples of analyses: B° decay to Dtt and 

D*TV 

Relevant quantities, sensitive to X, are formed 
from the kinematic variables of the particles de- 
tected and measured. These are typically dis- 
played in a histogram, or histograms. (Joint 
two-dimensional plots are also common. Some- 
times, but rarely, the data at this stage is a sin- 
gle number.) 

These distributions are then compared with the 
theoretical predictions, of which there may be 
several. One will be the predicted distribution 
if X is not present. Another may be the predic- 
tion if X is present in the amount, and with the 
properties, predicted by an expected theory such 
as the 'Standard Model' of Particle Physics. 
There may also predictions obtained within the 
framework of a particular model, but with one 
or more parameters adjusted to fit the data. 

An example of such a result is shown in Figure 1 
(taken from Q). In the top plot the phenomenon X 
is the decay of the B° meson to Dir, in the lower 
plot the decay to D*ir. The distributions show the 
invariant mass, which is the quantity given by 

mv = (j>) -(x> c ) a) 

where the sums run over the two final-state particles. 
If the two observed particles do indeed come from the 
decay of a B° particle then this quantity should be 
5.28 GeV/c 2 , though this is smeared out by experi- 
mental resolution. The plots show the predictions of 
a theory in which this decay does not occur (and all 
events are background) and also a prediction in which 
the decay is produced, with a normalisation adjusted 
to give the best fit to the data. The result of this 
fit gives the number of signal events, from which the 
branching ratio can be obtained (though in fact that 
was not done in this example). 

If that looks trivial, a harder example is the decay 
B — > 7r°7r°, taken from t 3J and shown in Figure 2. (To 
be fair, things are not quite as bad as this 1 dimen- 
sional plot implies.) 

In this confrontation of theory with experiment, one 
can then ask: is there any evidence for X or is the null 
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m ES (GeV/c ) 

Figure 2: Another example of an analysis: B° — ► 7r ( V° 

hypothesis unrefuted? Given that there is X, what is 
the best estimate for the normalisation (and perhaps 
other) parameter(s) involved in our model for XI Are 
these results compatible with the standard prediction 
for XI These are familiar statistical questions. 

1.3. Statistics in HEP 

From the above description we can bring out some 
features of the way statistics is used in HEP. 

Firstly, everything is a counting experiment. To 
measure a branching ratio or a cross section, one 
counts the number of events produced and observed. 
To measure the mass of a particle one uses a histogram 
where the number of entries in each bin is a random 
Poisson process. (The data of Figure 1 could be used 
to fit the mass of the B° meson, were it not already 
well known.) Poisson statistics is of paramount im- 
portance. Even the Gaussian (Normal) distribution 
plays its main role as the large N limit of the Poisson. 
(There are exceptions to this generalisation, but they 
occur in the details of the reconstruction of particle 
quantities.) 

This unpredictability is not due to any lack of 
knowledge on our part: not sampling error, or mea- 
surement error, or due to unconsidered effects. It is 
true and absolute randomness, driven by the funda- 
mental nature of quantum mechanics. We know that, 
for instance, a K® particle may decay into two charged 
pions or two neutral pions, with probabilities of 69% 
and 31% respectively. That is all we can ever know. A 
sample of K° particles will decay to ir + ir~ and 7r°7r° in 
a ratio of roughly 2:1 even if they are prepared abso- 
lutely identically - we have no hope of ever being able 
to say which ones are more likely to 'choose' one path 
rather than another. Likewise the timing of a decay is 
absolutely random in that the probability that a par- 
ticle existing at time t will decay before time t + 5t is 
a constant, independent of the value of t; there is no 
'ageing' process. 

But the Poisson distributions that result are just 
like any conventional Poisson process. These and 
other uncertainties, are (almost always) controlled 
and understood. These distributions have standard 
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deviations known to be >/~N. The Gaussian used for 
the signal distributions in Figure 1 is well established 
(it has a mean of 5.28 and a standard deviation of 
0.0025 GeV/c 2 ). 

So, in common with the other physical sciences, the 
distributions involved (signal, backgrounds) are given 
by functions known up to a few parameters - which 
can be fitted for. The approach to the data is not 
descriptive (identifying features, looking for trends) 
but prescriptive: the distribution is taken as having 
some functional form, and one has a pretty good idea 
as to what that functional form is, apart (possibly) 
from a few adjustable parameters. 

1.4. Unused Statistical Methods 

A consequence of this knowledge of uncertainty - 
the fact that we know what it is that we don't know - 
is that many techniques commonly used in the broad 
field of statistics are little used (or not used at all) in 
particle physics. 

Student's t is unknown. This is a technique used to 
handle small numbers of values from a distribution of 
unknown mean and unknown standard deviation, but 
our uncertainties come from known measurement er- 
rors. (If a measurement error is not known, a separate 
large-number determination is made.) The F test and 
ANOVA, tools for studying problems with unknown 
variances, are similarly of little use. The whole ex- 
perimental design field - Latin squares and similar 
techniques used to minimise uncontrollable effects - 
is not needed as such effects are not a problem. 

Another set of neglected techniques are those han- 
dling Time Series and Markov chains. Changes with 
time can be relevant in some studies, but it appears 
in them as another quantity to be measured and his- 
togrammed. The development with time of a particle 
is basically smooth, punctuated by radical transfor- 
mations (such as the decay of a particle to two or 
more lighter ones) which occur at random times. 

Non-parametric Statistics are also barely featured, 
as all these distributions, which are believed to be true 
idealisations of what 'really' happens, or at least good 
approximations to them, are parametrised. 

The notion of a Parent population is not helpful: 
a sample of particles is taken, but the randomness is 
(as stated earlier) inherent in the nature of particle 
behaviour and not produced by the sampling. If there 
is a parent distribution, it is an infinite set of particles 
produced under these conditions - all the events we 
might have seen. 

The point here is not that particle physics has noth- 
ing to learn from standard statistical techniques. The 
Statistician has many implements in their toolbox. 
Different fields of application will call for different 
tools; some of those heavily used in other fields are 
of less relevance in this one. 
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2. Tools 

Having seen that particle physics makes little use of 
some statistical tools, we take a more detailed look at 
the ones it does utilise. 

2.1. Monte Carlo Simulation 

Theoretical distributions for the quantitites being 
studied are predicted by quantum mechanics - per- 
haps with a few unknown parameters - and are often 
beautiful and simple. Angular distributions may be 
flat, or described by a few trigonometric terms; masses 
often follow a Cauchy function (which the particle 
physicists call the Breit-Wigner) , time distributions 
may be exponential, or exponential with a sinusoidal 
oscillation. 

These beautiful and simple forms are generally 
modified by unbeautiful and complicated effects 
(higher-order calculations in perturbation theory, or 
the fragmentation of quarks into other particles) . Fur- 
thermore the measurement and reconstruction process 
that the detector does for the particles is not com- 
pletely accurate or completely efficient. 

The translation from knowing the distributions in 
principle to knowing them in practice is done by 
Monte Carlo simulation. Particles are generated ac- 
cording to the original simple distributions, and then 
put through repeated random processes to describe 
the theoretical complications and then the passage of 
particles through the detector, including probabilities 
for colliding with nuclei in the beam pipe, slipping 
through cracks in the acceptance, or other eventual- 
ities. A complete software representation of all the 
experimental hardware has to be coded. The effects 
of the particles on the detector elements is simulated 
and the information used to reconstuct the kinematic 
quantities using the same programs that are run on 
the real data. This provides the full theoretical distri- 
bution function that the data is predicted to follow, 
albeit as a histogram rather than a smooth curve. 

These programs are large and slow to run. Signif- 
icant resources (both people and machines) are put 
into them. The generation of 'Monte Carlo data' is a 
significant issue for all experiments. Cases are known 
where data has been taken and analysed but results 
delayed because of lack of the correct Monte Carlo 
data 0|. 

2.2. The Likelihood 

Having the parametrised theoretical description of 
the distribution means the likelihood function is al- 
ways known, and it assumes an overwhelmingly im- 
portant position. Writing this function - where the 
Xi are the data and 9 the unknown parameter(s) 



L(xi,x 2 ...x N \6) =Y[P(xi\0) InL = 5^lnP(si|e) 

i 

the form p(x\0) is totally known, and L (or InL) fol- 
lows. 
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Figure 3: The log likelihood as a function of a parameter 

Having the likelihood function, the Maximum Like- 
lihood estimator is then easy to implement, and is very 
widely used. Even estimators like least-squares are, at 
least by some, 'justified' as being derivable from Max- 
imum Likelihood. Its (asymptotic) efficiency, and its 
invariance properties are desirable and useful. 

In some cases the ML estimate leads to an algebraic 
solution but in general, and in complex analyses, the 
physicist just maps out In L for their dataset as a func- 
tion of 8 and reads off the ML estimator from the peak, 
as can be done in Figure OH This also produces an in- 
terval estimate as part of the minimisation process. 
Following the value of In L until it falls off by \ from 
its maximum gives the 68% central confidence inter- 
val. Strictly speaking this is valid only for large N, 
but this restriction is generally disregarded. Perhaps 
we should not be so cavalier about doing so. 

Maximum Likelihood methods can also be used for 
functions with several parameters, as illustrated in 
Figure^] Confidence regions are mapped out by read- 
ing off the likelihood contours. This is done in many 
analyses and the MINUIT program [5( is widely used 
in exploring the likelihood and parameter space. 

2.3. Fitting Data 

Fitting the parametrised curve to the experimental 
data is done by several techniques. 

I) \ 2 using o~ — n i.e. minimising \ 2 — 
y\ nag ^jjg advantage that the minimi- 

sation can be done by differentiating and solving the 
normal equation, which is especially simple if / is lin- 
ear in 9. However the use of the observed number 
rather than the predicted number in the denominator 
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Figure 4: Contours of In L in two dimensions 

is recognised to lead to bias (downward fluctuations 
get an undue weight) and this cannot safely be used if 
n is small. (Actually in many cases what happens is 
that one of the bins has n = 0, and the physicist gets 
divide- by-zero messages and then starts to worry.) 

2) x 2 using cr 2 = / i.e. the predicted value 
rather than the actual number, avoids the bias (and 
the divide-by-zero problem) but gives nonlinear equa- 
tions. It still suffers from using a Gaussian probability 
as an approximation to a Poisson distribution and is 
thus not the 'real' maximum likelihood estimator. 

3) 'Binned Maximum Likelihood' uses the Poisson 
likelihood in each bin rather than the \ 2 - It is there- 
fore a proper Maximum Likelihood estimator. Effi- 
ciency is lost (only) if the bins are wider than the 
structure of the data. 

4) Full maximum likelihood does not use binning 
at all. It can be useful for very small event sam- 
ples. For large samples it becomes computationally 
intensive (as there is a sum over events rather than a 
sum over bins) though with today's computers this is 
hardly important. Perhaps a more significant factor 
for physicists is that it does not have the readily in- 
terpretable graphic image given by a histogram and 
fitted curve. 

2.4. Goodness of Fit 

Having found a fit, one has to judge whether to 
believe it. Whether the question is 'Does the curve 
really describe the data?' or 'Do the data really fit 
the curve' depends on one's point of view. 

The likelihood value does not contain the answer 
to this question. This appears counter-intuitive and 
many people have wrestled (unsuccessfully) to pro- 
duce ways that the likelihood can be used to say some- 
thing about the quality of the fit. 

The x 2 = J2i ( Vi ~^ Xi ^ 8 ' > J certainly does give a 



goodness of fit number. It is heavily used for GoF and 
2-sample tests: researchers may quote \ 2 or X 2 /Nd or 
the probability of exceeding this \ 2 ■ 

Alternative measures of goodness of fit have never 
really caught on. The Kolmogorov-Smirnov test is oc- 
casionally used - generally misleadingly, in my opin- 
ion. This is a totally robust test but pays the price for 
that by being weak. If you know anything about the 
data, e.g. that the numerical value of the parameter 
means something, then a more powerful test should 
be available. The KS test is being used to certify that 
distributions are in agreement when a more powerful 
approach would show up a difference. 

2.5. Toy Monte Carlo 

The 'Toy Monte Carlo' has emerged as a technique 
made possible by modern computing resources. Hav- 
ing obtained a result, it may be hard or impossible to 
obtain significance levels or confidence regions in the 
traditional analytic way, for instance if the likelihood 
function one is studying does not even plausibly re- 
semble a distorted parabola, but instead some shape 
with multiple maxima. 

As an alternative approach, starting with an esti- 
mate exp from the data, say {x\ . . . Xn}, how can one 
establish a confidence region? Consider any particular 
9. Use the known L(x\6) to generate a set of N values 
of x - an "experiment'. Use this in your estimator 
(whatever that is) to find a corresponding 0. Repeat- 
ing many times gives the probability that this 6 will 
give an estimate below (or above) the experimental 
one. This is just what the Neyman construction uses. 
To find a particular confidence region one has to ex- 
plore the parameter space until one finds the limits 
one wants. 



3. Topics 

Having explained the basic and generally agreed 
techniques used, there are a number of topics where 
advances are being made, or which are the subject of 
heated dicussion and argument, or both. 

3.1. Bayesian Probability 

The religious war which has been waged over the 
past few years has now cooled - although some iso- 
lated zealots remain on both sides. The 'frequen- 
tists' have come to accept that the use of Bayesian 
techniques can be illuminating and helpful, and some- 
times provide more useful information than a frequen- 
tist confidence level, especially for measurements of 
bounded parameters (e.g. masses). The 'Bayesians' 
are recognising that Bayesian confidence levels will not 
totally replace the use of frcquentist levels, and that 



MOAT002 



G 



PHYSTAT2003, Stanford Linear Accelerator Center, September 2003 



they do have to take on board the issue of robustness 
(or otherwise) under changes of prior. 

A real benefit of this debate has been to bring 
the subject out into the open. The classic statistics 
texts [U0! from which many particle physicists first 
learned the subject, slide swiftly between the frequen- 
tist and Bayesian concepts of probability, never really 
acknowledging that they are using two very different 
quantities. 

3.2. Small Signals and Confidence 
Regions 

The 'Energy Frontier' is a cutting edge of parti- 
cle physics: new, more powerful, accelerators open 
up new areas for investigation and new particles are 
discovered. Another cutting edge is the 'Luminos- 
ity Frontier': the discovery that processes hitherto 
thought to be impossible do actually occur, albeit very 
rarely. The discovery of CP violation ffl: that the 
probability of the decay K° L — > 7r + 7r~ was not zero but 
0.2%, was enormously important despite the smallness 
of the figure. Many of today's experiments are look- 
ing for phenomena which are known to be exceedingly 
rare, at the parts-per-million level at best. 

Although the implications can be spelt out quite 
simply and dramatically - 'If the AMS experiment 
sees even one 12 C nucleus, our entire view of the uni- 
verse will change.'- in practice things are not so clear- 
cut because of the presence of background. Also one 
has to be able to handle not just the dramatic discov- 
eries, but the much more frequent useful analyses that 
make no discovery but push back the limits and the 
region in which any discovery may be made. 

An experiment that sees no events will note the 
standard result from Poisson statistics that an ob- 
served number of zero translates to a limit on the true 
value of less than 3 events, with 95% confidence. This 
can then be converted (using the figures for this par- 
ticular experiment) into a limit on the branching ratio 
or cross section for the process concerned, and then 
possibly into a limit on a mass or coupling constant. 
If there is an expected background for this process 
equivalent to, say, 0.2 events, then the amount for the 
branching ratio limit is reduced to 2.8. But this clearly 
has problems: suppose the predicted background were 
3.1 and no events were observed (unlikely but not im- 
possible), what can one then say about the limit? 

There has been a lot of activity and discussion 
recently in this area. Indeed it sparked off the 
workshop 9] of which this conference is the succes- 
sor. The standard frequentist (Neyman) construc- 
tion may result in statements about results in the 
non-physical region (here, a negative number of sig- 
nal events) which, though statistically correct, appear 
nonsensical. Bayesian methods avoid this problem, 
as does the frequentist technique proposed by Feld- 



man Cousins |l0j | which switches smoothly and auto- 
matically between quoting central and one-sided con- 
fidence regions. 

3.3. When to Claim a Discovery? 

Another area of discussion is over the form of re- 
porting non-zero signals. When the number of signal 
events is much larger than the expected background, 
or a fitted parameter is significantly different from the 
theoretical prediction, then clearly the experiment can 
claim a discovery. If the numbers or parameter values 
are compatible, the experiment quotes an upper limit. 
But there is an area in between where the probability 
of the null hypothesis giving the result is small enough 
to be interesting, but not so small as to be completely 
negligible. The experiment must not be rash, phoning 
the New York Times with a discovery which turns out 
to be a statistical fluctuation, nor must it be too cau- 
tious or the subject can never progress. Such results 
are bound to occur - the probability that an exper- 
iment will produce a value in this region is by defi- 
nition small-but-not-negligible, or better. Given the 
large number of busy experiments reporting results, 
this is a real problem. 

Some experiments have policies such as 4<r for 'evi- 
dence for', 5cr for 'discovery of - significance levels are 
often presented in terms of the equivalent discrepancy 
in standard deviations. Is it possible to report a two- 
sided result (as the Feldman Cousins technique will 
sometimes produce) and yet not claim a discovery? 
'We report with 95% confidence that the branching 
ratio lies in the range (2.3 to 3.4) 10~ 6 but we're not 
actually claiming to have seen it.' Such 'discoveries' 
are reported in a way which must be affected by the 
prior (subjective) probability, in exactly the way the 
Bayesians describe. Statistically identical data on the 
decays B + — > 7r + 7r° and B + — > n~n Q would be re- 
ported completely differently. 

3.4. Blind Analysis 

In recent years particle physicists have become 
aware of practitioner bias. This has been fuelled par- 
ticularly by reports from the Particle Data Group, 
which has the job of reporting and combining the mea- 
surements of particle properties [TT| . who show how 
some values change significantly over time, but never 
by more than one standard deviation. Another source 
of disquiet was the Electroweak measurements from 
LEP and the SLC which agree with each other and 
with the Standard Model far too well with a x 2 P er 
degree of freedom well below 1 12] . 

This practitioner bias is against claiming differences 
from the null hypohesis. The experiment template 
presented in section Tl . 21 often continues 
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• Extract result, usually by fitting parametrised 
distribution(s) to data. 

• Compare your result with that of accepted the- 
ory and/or other experiments. 

• If it disagrees, look for a bug in your analysis. 
You will probably find one. Keep searching and 
fixing until the agreement is acceptable. 

The mistake in method is that the experimenter 
stops looking for bugs when they have agreement, not 
when they honestly believe that all (substantial) bi- 
ases are accounted for. To guard against this the data 
can be 'blinded'. There are two techniques used, cov- 
ering two types of situation 

• In the extraction of a result, this can be encoded 
by some unknown offset. 

• Choosing the cuts which select the data is done 
on Monte Carlo data, or on real data in side- 
bands - regions close to but not actually includ- 
ing the region where the signal is expected. Oth- 
erwise the temptation to nudge a cut slightly to 
include a few more events is too great. 

3.5. Systematic Errors 

In the early days of particle physics, the 50s and 60s, 
a typical experiment would get handfuls of events - a 
few hundred if lucky - from painstaking analysis of 
bubble chamber pictures. Statistical errors were thus 
~ 10% and were so large that the effect of systematic 
uncertainites was generally small. 

In the 70s and 80s, the development of counter ex- 
periments led to event samples in the tens of thou- 
sands. Statistical errors were now at the per cent level, 
and systematic errors began to be more important. 

The current generation of experiments - the Z fac- 
tory at LEP, the B factories, Deep Inelastic Scattering 
at HERA - deal with millions of events. Statistical er- 
rors are at the level of ~ 0.1% and we have learned 
how to talk about 'parts per mille'. 

Systematic errors (uncertainties in factors system- 
atically applied in the analysis) can no longer be 
fudged. The word 'conservative' has been grossly 
overused in this context. It sounds safe and reas- 
suring; in practice it is usually a sign of laziness or 
cowardice. The experiment perhaps cannot be both- 
ered to evalute an uncertainty and makes a guess, and 
then it inflates that guess to cover the possibility that 
they'll be caught out, and calls it a 'conservative' es- 
timate of the systematic error. 

Particle physicists also confuse the evaluation of 
systematic errors with overall consistency checks. 
There is bad practice being spread to and between 
graduate students. They will identify all the calibra- 
tion constants and parameters that contribute to the 



final result and vary those by their appropriate error, 
and fold the resultant variation into the systematic 
error. This is correct procedure. But they will also 
vary quantities like cut values, which should not in 
principle affect the result, by some arbitary amount 
and then solemnly fold those resulting variations into 
the systematic error. This is nonsense. Looking at 
what happens when you change a cut value is a good 
and sensible thing: a (say) looser cut will give a higher 
efficiency and a higher background and thus more ob- 
served events, but after correcting for the new effi- 
ciency and background the result should be compati- 
ble with the original. This is a useful check that one 
understands what's going on and that the analysis is 
consistent. But it does not feed into a numerical un- 
certainty. 

3.6. Unfolding 

Measurements of the properties of particles in 
events are made with finite resolution, so the plots of 
these quantities, and functions of these quantities, are 
'smeared out'. Events move between histogram bins. 
Sharp peaks become broad, edges become slopes. 

The recovery of the original sharp distribution from 
the observed one is known as 'unfolding'. This is an 
alternative use of the Monte Carlo simulation process: 
rather than compare the data with a theoretical pre- 
diction smeared by Monte Carlo simulation, one com- 
pares the original theory with the de-smeared data. 
Clearly this is preferable, if it can be done, as the un- 
folding process depends only on the experiment and 
not on the original theory, and so once unfolded the 
data can be compared with any prediction. 

It looks a simple problem: given an original distri- 
bution as a histogram, the probability of migration 
from any bin i to any bin j, Pji, can be estimated 
from a Monte Carlo sample (this includes the prob- 
ability that it may not be accepted: ^jPji < 1)- 
The matrix is inverted, and then applied to the data 
histogram to give the reconstructed original. 

Unfortunately it is not at all simple • I* 1 the ma- 
trix inversion the errors on the Pji from finite statistics 
have devastating consequences and produce unrealis- 
tic results. There is a lot of activity in handling this in 
a sensible way, and in investigating other approaches, 
such as Maximum Entropy techniques. 

3.7. Combining Results 

The combination of compatible measurements with 
different errors is straightforward. However results 
are sometimes incompatible, or marginally compati- 
ble. But something must be done with the results, 
as the community needs a way of using the combined 
number. Indeed it is the responsibility of the Particle 
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Data Group [ll| to combine measurements and form 
'world average' results in a meaningful way. 

There is also a problem in combining limits. If two 
experiments report 95% confidence level upper limits 
of, say, 0.012 and 0.013, how can one combine these 
two measurements? This question was put forcefully 
by the Higgs searches at the end of the LEP run. 
The four experiments reported results separately com- 
patible and possibly marginally suggestive of a signal 
from a Higgs boson of mass around 114 GeV/c 2 . Did 
four possibles make a probable? The answer to that 
statistics question determined whether or not LEP 
would run another year, at a cost of millions not only 
in power bills but in its impact on the construction 
schedule for the LHC. The CERN management de- 
cided that the answer in this case was 'no'. History 
will be their judge. 

In combining experiments the likelihood function 
contains much more information than a simple limit, 
or value and error. There is a suggestion that these 
should be routinely published, and we are probably 
going to see that happening a lot in the future. 

3.8. Multivariate Classification 

The classification of events (usually 'signal' and 
'background') and particles (pion, kaon...) by means 
of a cut on a discriminator variable is a basic hypoth- 
esis testing problem. However there may be several 
variables, each containing useful information, and the 
best choice will be made by combining these in some 
way. 

The Fisher Discriminant has been re-discovered as 
a technique which is good if the means of distributions 
differ between the two samples. The Neural Network 
(feed-forward 'perceptron' configuration) has become 
a standard item in the toolbox which can handle more 
general differences, and there are many developments 
going on in this area. 

The use of cuts is deeply engrained. In many cases 
it is simple and appropriate. However in cases where 
there are no clean boundaries it may be better to 
consider all events, weighting them according to their 
signal-like or background-like nature. 

4. Conclusions 

I have given several talks on 'Statistics for Particle 
Physicists' but 'Particle Physics for Statisticians' has 
been a new and interesting experience. This has been 
a very broad view. Particular topics will be considered 
in detail in the subsequent talks in this conference, 
in both plenary and parallel sessions. Hopefully the 
account here will provide you with a map which will 
help you place them in context. 
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